Design and Analysis of the NIPS 2016 Review Process

نویسندگان

  • Nihar B. Shah
  • Behzad Tabibian
  • Krikamol Muandet
  • Isabelle Guyon
  • Ulrike von Luxburg
چکیده

Neural Information Processing Systems (NIPS) is a top-tier annual conference in machine learning. The 2016 edition of the conference comprised more than 2,400 paper submissions, 3,000 reviewers, and 8,000 attendees, representing a growth of nearly 40% in terms of submissions, 96% in terms of reviewers, and over 100% in terms of attendees as compared to the previous year. In this report, we analyze several aspects of the data collected during the review process, including an experiment investigating the efficacy of collecting ordinal rankings from reviewers (vs. usual scores aka cardinal rankings). Our goal is to check the soundness of the review process we implemented and, in going so, provide insights that may be useful in the design of the review process of subsequent conferences. We introduce a number of metrics that could be used for monitoring improvements when new ideas are introduced.

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Advance and Green Manufacturing Technology Adoption: Classification and Literature Review of Issues

Purpose – The actual benefits of the AMT solidarity to the manufacturing system, classified as systemic, will only be acquired and identified if the current design and organizational structure become compatible with the change being introduced. The purpose of this paper is to present a review for the organizational design development related to AMT assumption and the green manufacturing (GM) in...

متن کامل

A Review on the Extent of Urban Design Intervention in Iran’s Architectural Structures for Interaction with Architectural Innovation

The intervention of urban design in Irans architectural structures has been for many years a challenging issue for architects and urban designers over their professional realm. The preservation of architectural structures in Iran has always resorted to extremes. This control and direction has sometimes resulted in exclusion of architecture position and has sometimes, once faced with lack of pro...

متن کامل

Determination of Cost Parameters in Total Design for Manufacturing Parts

Design process is a key process in forming of product’s cost and if we notify to total design we will see all of the effective parameters in design activities. Design process is contented design planning, design inputs, design outputs, design review, design verification, design validation and design changes. This approach will help us to identify the cost parameters for managing, controlling an...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

ثبت نام

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

عنوان ژورنال:
  • CoRR

دوره abs/1708.09794  شماره 

صفحات  -

تاریخ انتشار 2017